A known method of avoiding excessive “wait states” in a processor—wherein the processor is idly waiting for a memory operation to complete rather than doing useful work—is the use of a dedicated high-speed local memory to the processor. In various architectures, this high-speed memory may take the form of one or more caches or L1, L2, or other local memories, each with its own particular advantages and uses. Throughout this Specification, all such local, high-speed memories are referred to collectively as “caches.”
Such local memories are, however, only useful when needed data can be written to or read from the local memory. A “cache miss” occurs specifically when the processor needs a value from a particular memory location, and that memory location has not been loaded into cache. In that case, the memory subsystem may need to perform a cache fill to fetch the needed value from memory, during which time the processor may be at least partly idle.